MaplePrimes - Maplesoft Blog - Posts by Dr. John May

About That 4x4 Matrix Multiplication Result

John May — Thu, 22 May 2025 21:02:53 Z

Last week, DeepMind announced their new AlphaEvolve software along with a number of new results improving optimization problems in a number of areas. One of those results is a new, smaller, tensor decomposition for multiplying two 4 × 4 matrices in 48 scalar multiplications, improving on the previous best decomposition which needed 49 multiplications (Strassen 1969). While the DeepMind paper was mostly careful about stating this, when writing the introduction, the authors mistakenly/confusingly wrote:

For 56 years, designing an algorithm with rank less than 49 over any field with characteristic 0 was an open problem. AlphaEvolve is the first method to find a rank-48 algorithm to multiply two 4 × 4 complex-valued matrices

This gives the impression that this new result is the way to multiply 4 × 4 matrices over a field with the fewest number of field multiplications. However, it's not even the case that Strassen's 1969 paper gave a faster way to multiply 4 × 4 matrices. In fact, Winograd's 1968 paper on computing inner products faster could be applied to 4 × 4 matrix multiplication to get a formula using only 48 multiplications. Winograd uses a trick

which changes two multiplications of ab's into one multiplication of ab's and two multiplications involving only a's or only b's. The latter multiplications can be pre-computed and saved when calculating all the innerproducts in a matrix multiplication

    # i=1..4, 4*2 = 8 multiplications
    p[i] := -A[i, 1]*A[i, 2] - A[i, 3]*A[i, 4];
    # 4*2 = 8 multiplications
    q[i] := -B[1, i]*B[2, i] - B[3, i]*B[4, i];

    # i=1..4, j=1..4, 4*4*2 = 32 multiplications
    C[i,j] = p[i] + q[j] 
            + (A[i,1]+B[2,j])*(A[i,2]+B[1,j])
            + (A[i,3]+B[4,j])*(A[i,4]+B[3,j])

It is simple to verify that C[i,j] = A[i,..] . B[..,j] and that the above formula calculates all of the entries of C=A.B with 8+8+32=48 multiplications.

So, if Winograd achieved a formula of 48 multiplications in 1968, why is the DeepMind result still interesting? Well, the relative shortcoming of Winograd's method is that it only works if the entries of A and B commute, while tensor decomposition formulas for matrix multiplication work even over noncommutative rings. That seems very esoteric, but everyone's favorite noncommutative ring is the ring of n × n matrices. So if a formula applies to a matrix of matrices, that means it gives a recursive formula for matrix multiplication - an tensor decomposition formulas do just that.

The original 1969 Strassen result gave a tensor decomposition of 2 × 2 matrix multiplication using only 7 scalar multiplications (rather than the 8 required by the inner product method) and leads to a recursive matrix multiplication algorithm using O(N^log[2](7)) scalar multiplications. Since then, it has been proved that 7 is the smallest rank tensor decompostion for the 2 × 2 case and so researchers have been interested in the 3x3 and larger cases. Alexandre Sedoglavic at the University of Lille catalogs the best tensor decompositions at https://fmm.univ-lille.fr/ with links to a Maple file with an evaluation formula for each.

The previous best 4 × 4 tensor decomposition was based on using the 2 × 2 Strassen decomposition recursively (2 × 2 matrix of 2 × 2 matrices) which led to 7 2 × 2 matrix multiplications each requiring 7 scalar multiplications, for a total of 49. The new DeepMind result reduces that to 48 scalar multiplications, which leads to a recursive algorithm using O(N^log[4](48)) scalar multiplications: O(N^2.7925) vs. O(N^2.8074) for Strassen. This is a theoretical improvment over Strassen but in 2024 the best known multiplication algorithm has much lower complexity: O(N^2.3713) (see https://arxiv.org/abs/2404.16349). Now, there might be some chance that the DeepMind result could be used in practical implementations, but its reduction in the number of multiplications comes at the cost of many more additions. Before doing any optimizations I counted 1264 additions in the DeepMind tensor, compared to 318 in the unoptimized 4×4 Strassen tensor (which can be optimized to 12*7+4*12=132). Finally, the DeepMind tensor decomposition involves 1/2 and sqrt(-1), so it cannot be used for fields of characteristic 2, or fields without sqrt(-1). Of course, the restriction on characteristic 2 is not a big deal, since the DeepMind team found a 4 × 4 tensor decomposition of rank 47 over GF2 in 2023 (see https://github.com/google-deepmind/alphatensor).

Now if one is only interested in 4 × 4 matrices over a commutative ring, the Winograd result isn't even the best possible. In 1970, Waksman https://ieeexplore.ieee.org/document/1671519 demonstrated an improvement of Winograd's method that used only 46 multiplications as long as the ring allowed division by 2. Waksman's method has since been improved by Rosowski to remove the divisions see https://arxiv.org/abs/1904.07683. Here's that nice compact straight-line program that computes C = A · B in 46 multiplications.

And, here attached are five Maple worksheets that create the explicit formulas for the 4 × 4 matrix methods mentioned in this post, and verify their operation count, and that they are correct.

Strassen_444.mw Winograd.mw DM_444.mw Waksman.mw Rosowski.mw

If you are interested in some of the 2022 results, I posted some code to my GitHub account for creating formulas from tensor decompositions, and verifying them on symbolic matrix multiplications: https://github.com/johnpmay/MapleSnippets/tree/main/FastMatrixMultiplication

The clear followup question to all of this is: does Maple use any of these fast multiplication schemes? And the answer to that is mostly hidden in low level BLAS code, but generally the answer is: No, the straightforward inner-product scheme for multiplying matrices optimized memory access thus usually ends up being the fastest choice in most cases.

Announcing the MaplePrimes Art Gallery

John May — Mon, 21 Oct 2024 16:26:22 Z

With the 2024 Maple Conference coming up this week, I imagine one of two of you have noticed something missing. We chose not to have a Conference Art Gallery this year, because we have been working to launch new section of MaplePrimes: The MaplePrimes Art Gallery. This new section of MaplePrimes is designed for showing off your Maple related images, in a gallery format that puts the images up front, like Instagram but for Math.

To get the ball rolling on the gallery, I have populated it with many of the works from past years' Maple Art Galleries, so you can now browse them all in one place, and maybe give "Thumbs Ups" to art pieces that you felt should have won the coveted "People's Choice Award".

Once you are inspired by past entries, we welcome you to submit your new artwork! Just like the conference galleries, we are looking for all sorts of mathematical art ranging from computer graphics and animations, to photos of needlework, geometrical sculptures, or almost anything! Art Gallery posts are very similar to regular MaplePrimes posts except that you are asked to supply an Image File and a Caption that will displayed on the main Gallery Page, the post itself can include a longer description, Maple commands, additional images, and whatever else you like. See for example this Art Gallery post describing Paul DeMarco's sculpture from the 2021 Maple Conference Gallery - it includes an embedded worksheet as well as additional images.

I can't wait to see what new works of art our MaplePrimes community creates!

Last Call for Maple Art!

John May — Mon, 11 Sep 2023 20:36:26 Z

we have recieved lots of great sumissions, but we want your great submission and now you have more time.

The deadline for submissions to the Art Gallery and Showcase for the 2023 Maple Conference is rapidly approaching. We really want to see your art! It doesn't have to be incredibly impressive or sophisticated, we just want to see what our community can create! If you've been working on something or have a great idea, you still have a few days to get it together to submit.

Submission can be made by email to gallery@maplesoft.com but be sure to visit the visit our Call for Creative Works for details on the format of the submission.

Maple Conference 2023: Calling all Artists!

John May — Thu, 08 Jun 2023 23:32:16 Z

If you've seen Paulina's announcement then you know that we are once again holding a virtual Maple Conference this year. As well, we are once again going to have a virtual gallery featuring artwork and creative projects submitted by the Maple community!

Last year we had a number of great submissions to our Maple Art Gallery and our Maple Learn Creative Showcase. These were our excellent prize winners.

From left to right we have A visualization of all the primitive roots of 10037 created by Simon Plouffe, winner of the Judge’s Choice, Mother’s Day Rose created with Maple plots by Greg Wheaton, winner of the People’s Choice, and Mona Lisa in Maple Learn created by Paul DeMarco (with help from Leonardo DaVinci), the winner of the People’s Choice for the Maple Learn Showcase.

This year we are expanding the Gallery into two collections to encourage more people to submit. They are

The Art Gallery - A small gallery to highlight high effort, mathematically interesting works (with stricter criteria)
The Creative Works Showcase - A larger showcase for nearly any interesting visual works created with Maplesoft products like Maple Learn and Maple

Feel free to submit nearly anything cool for the Creative Works Showcase, if we find it particularly impressive we might even ask you to let us consider it for the gallery. Also, do not be intimidated by the title "Art Gallery" we're looking for anything that has taken some artistic effort and tells a mathematical story.

For more information on critera and how to submit, please visit our Call for Creative Works. The important deadline to know is the September 14th deadline for submission of works with virtual gallery reception and awards ceremony durring the conference October 26-27.

I look forward to seeing all the submissions for the Maple community again this year!

Maple Conference 2022 Creative Works Deadline Next Week

John May — Tue, 13 Sep 2022 18:08:30 Z

This is a friendly reminder that the deadline for submissions for this year's Maple Conference Creative Works Exhibit is fast approaching!

If you are looking for inspiration, you can take a look at the writeup of the works that were featured last year in this write up in the most recent issue of Maple Transations.

Also, don't forget that you can also submit art made in Maple Learn for a special exhibit alongside the main gallery.

Maple Conference 2022: Call for Creative Works

John May — Tue, 21 Jun 2022 22:05:54 Z

Hi Maple Users

As I hope you have already heard, Maplesoft is having our Maple Conference again this fall. And that means that

Last year we had many great submissions and you can still read about them in detail on the 2021 conference site. Some of the featured works were excellent Maple visualizations, including a special prize for a student contribution by Avek Dongol (center).

But we also featured a number of impressive physical works, including the people's choice winning wood carving by Paul DeMarco (left), and the judges' choice winning cross stitch by Bridjet Lee and Curtis Bright (right).

This year, we are again looking to fill our virtual exhibition with all types of mathematical art, ranging from computer graphics and animations, to needlework, geometrical sculptures, or almost anything you can come up with. Surprise us!

The full announcement can be found at the Maple Conference Art Gallery page. We would like to have all submissions by September 22nd so that we can review and finalize the gallery before the conference begins November 2nd.

I can't wait to see what everyone sends in this year!